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ASDC Introduction 


The Atmospheric Science Data Center (ASDC) at NASA 
Langley Research Center is responsible for the ingest, 
archive, and distribution of NASA Earth Science data in 
the areas of radiation budget, clouds, aerosols, and 
tropospheric chemistry. The ASDC specializes in 
atmospheric data that is important to understanding the 
causes and processes of global climate change and the 
consequences of human activities on the climate. The 
ASDC currently supports more than 44 projects and has 
over 1 ,700 archived data sets, which increase daily. 
ASDC customers include scientists, researchers, federal, 
state, and local governments, academia, industry, and 
application users, the remote sensing community, and the 
general public. 


ICARTT Machine Readability Issues 


Variable names are not standardized 

Different instrument Principal Investigators (PI) may name the 
same variable differently in a mission 
Different names exist for the same variable in different 
missions 


CCN 




Cloud_lnteg_Num_Density AircraftAltitude 


NCCN 


NCAR_CH20_lsec(pptv) 


Number Concentration 


CCN Concentration 


NCAR_CH2O_10sec(pptv) 


NCAR_CH20(pptv) 


CH20_pptv 


H2CO_ppbv 


num concentration CAS 


CloudNGT30 


cloud_number_density 


N TOT 


nCAS 


NCAR CH20 DFG 


ALTITUDE GPS 


PALT 


HGM232 


GGALTC 


GpsAlt 


FMS ALT PRES 


Various names used for the same variable across different missions, 

Date/time recording is inconsistent 

• No requirements for naming time variables 

• Not always a simple way of determining what the variable is 
actually measuring (start, stop, mid) 

File structure varies greatly 


52 1001 

Collins, Don 

Texas A&M University 

Tandem Differential Mobility Analyzer 

from the Cl 30 

NCAR MILAGRO Mission 2006 
1 1 

2006 03 04 2007 02 27 
0 

Start_UTC, sec 
14 

11111111111111 
-9999 -9999 -9999 -9999 -9999 - 
9999 -9999 -9999 -9999 
End_UTC, sec 
MidJJTC, sec 
Latitude, fractional degrees 
Longitude, fractional degrees 


41,1001 

Anderson, Bruce E. 

NASA Langley 

Black Carbon Number Density 
Measurements with a SP-2 
DISCOVER-AQ 
1,1 

2014,07,20,2015,09,09 

0 

UTC_start,Secs after midnight, Time 
of acquisition 
3 

1,1,1 

-9999,-9999,-9999 

UTC_end, Secs after midnight, Time 

of acquisition 

UTC_mid,Secs after midnight, Time 


Airborne Tropospheric Chemistry Studies 


NASA conducts airborne tropospheric chemistry studies, and has for over three decades. These field campaigns 
generate a great wealth of observations, including a wide range of the trace gases and aerosol properties. Even 
though the spatial and temporal coverage is limited, the aircraft data offer high resolution and comprehensive 
simultaneous coverage of many variables, e.g. ozone precursors, intermediate photochemical species, and 
photochemical products. The recent NASA Earth Venture Program has generated an unprecedented amount of 
aircraft observations in terms of the sheer number of measurements and data volume. The ASDC Toolsets for 
Airborne Data (TAD) design meets the user community needs for aircraft data for scientific research on climate 
change and air quality issues. 


Working with ICARTT 


To compensate for the ICARTT file issues, files are first converted to an “idealized” format. The 
metadata is then stored in a comprehensive postgreSQL database. This is a three step process 

Step 1. Map all metadata to an “extended map”. Includes mapping each PI variable name to a common 
name. 


Column Name Description 


mission 

platform 

datajd 

pi_variable_name 

units 

description 

needed or not 
needed (1 or 0) 

common_name 

UNC variable 

LOD variable 

Flag var 

var_type 

scan_name 

show_data 

array_size 

arrayjndex 

ui_name 

group_def 

output_name 

instrument 

column_name 

orig_dataid 


The mission name. 

The platform associated with the mission. 

The data ID from which the variable comes. 

The variable name as listed in the file header. 

The units associated with the variable. These have been standardized. 

The variable description. 

Some variables will not be dealt with at all by TAD, so they will not be harvested. These are marked as 0. Everything else is marked as 1. 

The common name, including the category index, associated with the variable. #:#:#(#)_CommonName 
The column name of the uncertainty variable associated with this variable or N/A. 

The column name of the LOD variable associated with this variable or N/A. 

The column name of the data flag variable associated with this variable, if one exists. This also includes the good and bad data flags as such: 
VarName(GoodFlags/BadFlags). If there are multiple flags for either, they are semi-colon delimited. 

The variable type. 

Some variables used in the files have names that are difficult for the harvester script to parse. This column is the corrected name that the variable will be 
given in the idealized ICARTT file by the filer program. 

Not all variables are available to order directly. These are marked as 0, meaning they will not appear in the user interface. There are also default variables 
outputted with each file. These are marked as 2. All other variables are marked as 1. 

If this variable is part of an array, this gives the number of elements in the array. If not, 0. 

If this variable is part of an array, this gives the index of this variable in the array, starting at 0. If not, 0. 

The name that will be used for the variable on the Ul. Typically these are the same as the PI variable name, but occasionally it will be different in order to 
better distinguish the variable. One major example is when all array variables are grouped under one name, this column will have that group name listed 
under the first individual variable, and all others will be marked as N/A. 

A description for any group of variables marked as one, or N/A. 

The name that will be used for the variable in the output file. Typically this is the Ul name with "_PILastName" appended to the end. Occasionally some 
extra information is added before the last name to ensure that there are no repeated variable names in the output files. 

The instrument used to measure this variable. 

In some of the older files, the variable name does not match between the header and the data columns. To make it easier for the filer, the variable name 
from the data columns is stored here. 

Certain data IDs for the original files have been broken up based on any additional comments. For example, files from the data ID NCAR-CFI20-lMin and 

NCAR-CFI20-lSec are both originally from NCAR-CFI20, but as both can occur on the same day they are distinguished by _lMin or _lSec before the file 

extension. This column has the original data ID. 




(ill. 


Optical 

Properties 


J I Meteorological/ 
Values^ Navigational 


Radiation 

Variable 


Trace 

Gases 


Meteorological 


Navigational 


Carbon/H Y dr| Ha | ocarbon5/H |Hydrogen/0| N , t 
ocarbon | a | ogens I xygen | 


Other ■ sulfur 
Gases “ 


Alkanes 


Alkenes/A 

Ikynes 


Aromatics 


Oxygenates/A 

cids 


Above: Extended map format 


The Variable Common Naming System links the non- 
standardized PI variables. It consists of six groups based on 
the physical and chemical properties of the measurements. 
This system is scalable to properly handle future 
measurements and was created in consultation with GCMD 
and the airborne community at large. 

• Step 3. Extract metadata to the database 


Step 2. Convert to the idealized format via the 
automated filer program 


L 


Airborne 
Field Study 
Data 

Repository 


Data 

Ingest 

System 


Data 

Archive 

Database 


Toolsets 

for 

Airborne 

Data 


User Input 
and 
Output 


variable_category_index 

character(8) 


level_one_def 

text 

level_two_def 

text 

level_three_def 

text 

frequent_variable 

text 



aliases 


serialjd 

integer 


-4 variable_category_index 

character(8) 


L» commonname 

text 


alias 

text 



variable_category_index 

character(8) 

commonname 

text 

definition 

text 

formula 

text 

mwaw 

text 

ui_show 

integer 

associated 

text 


file info 


Raw 

ICARTT 


Common 

Variable 

Mapping 

Idealized 

ICARTT 

Mapping 


Idealized 

ICARTT 


Database 

Loader 


Above: Workflow for the automated filer program 


file_mission_id 

integer 

file pi id 

character(IO) 

file_data_id 

character(IO) 

file_flight_id 

character(IO) 

working_file_path 

text 

raw_file_path 

text 

headerjines 

integer 

file_volume 

integer 

num_file_volumes 

integer 

num_special_comment_lines 

integer 

revision 

text 

revision comments 

text 

utc_data_revision 

date 

utc_data_start_acq 

numeric 

utc_data_stop_acq 

numeric 

datajnterval 

integer 

start_time_label 

text 

stop_time_label 

text 

mid_time_label 

text 

ecs_category_id 

character(7) 


mission info 


mission 


pi_variable_info 


integer 

text 

text 


platform 
mission start date date 


pi_mission_id 

integer 

pi_id 

character(IO) 

pi_name 

text 

pi_org 

text 

pi_contact_info 

text 


fiightjnfo 

flight_mission_id 

integer 

flight_id 

character(IO) 

flight_date 

date 

flight_start_utc 

numeric 

flight_stop_utc 

numeric 


data id info 


data_mission_id 

integer 

datapijd 

character(IO) 

data_id_code 

character(IO) 

datajd 

text 

data_src_description 

text 

data_mgr_contact_info 

text 


ecs info 


variable_mission Jd 

integer 

variable_pijd 

character(IO) 

r»variable_data_id 

character(IO) 

serialjd 

integer 

pi_variable_name 

text 

units 

text 

description 

text 

commonname 

text 

variable_category_index 

character(8) 

instrument 

text 

lodjabel 

text 

uncjabel 

text 

data_flagjabel 

text 

good_data_flag 

text 

baddataflag 

text 

variable_type 

text 

array_size 

integer 

arrayjndex 

integer 

show_data 

integer 

missing_data_flag 

text 

scalejactor 

numeric 

llod_flag 

text 

ulod_flag 

text 

ui_name 

text 

groupdescription 

text 

outputname 

text 


category Jd 

character(7) 

category_desc 

text 


Above: Database schema 



Toolsets for Airborne Data 


TAD is focused on in situ observational data, 
which represent the majority of the airborne 
measurements in the Atmospheric 
Composition Focus Area 
TAD draws on aircraft data holdings at the 
ASDC to create a data discovery tool that 
generates on-the-fly weighted averages of 
derived value-added products for researchers. 
Automated parsing tools convert ICARTT files 
to an idealized format for TAD ingest 



Idealized ICARTT Format 


The idealized ICARTT format, a restructured ICARTT 
Data file, improves machine readability to sustain the 
TAD system. The idealized format lessened the 
complexities with Airborne data. The advantages to 
the idealized format are given below. 


Consistent delimiter for scale factors, missing data 
flags, variables, and data. 

Data interval of 0 or 1 only. 

Time variables always reported for start, stop, and 
mid. Null values used as placeholders when 
necessary. 

Short name, unit, and long name listed for all non- 
time dependent variables. 

Stricter rules for variable names, i.e. no commas or 
spaces 

Standardized delimiters for LOD flags. 

All normal comments listed. N/A used where 
necessary. 

Revision information listed chronologically. 
Guaranteed variable consistency between header 
and columns 


More Information 


Special Thanks 




NOAA ESRL 
GCMD 
WWW-AIR 


https://tad.larc.nasa.gov 



































